Replace Manual Coding of Customer Survey Comments with Text Mining: A Story of Discovery with Text as Data in the Public Sector

A common approach to analyzing open-ended customer survey data is to manually assign codes to text observations. Basic descriptive statistics of the codes are then calculated. Subsequent reporting is an attempt to explain customer opinions numerically. While this approach provides numbers and percentages, it offers little insight. In fact, this method is tedious and time-consuming and can even misinform decision makers. As part of the Alberta Government’s continual efforts to improve its responsiveness to the public, the Alberta Parks division transitioned from manual categorization of customer comments to a more automated method that uses SAS Text MinerTM. This switch allows for faster analysis of unstructured data, and results become more reliable through the consistent application of text mining. INTRODUCTION Alberta’s provincial parks system protects more than 27,600 square kilometres, or approximately 4.2 per cent of the province. This area is larger than Hawaii. The 478 sites in the Alberta Parks system offer a rich diversity of opportunities and uses. Some parks are designed for recreation, but many others support both conservation and recreation activities. There are 250 campgrounds with nearly 14,000 campsites in the Alberta Parks system, utilized by approximately 1.5 million campers annually. Since 2002, a province-wide Camper Satisfaction survey has been conducted at parks that offer camping. The purpose of this survey is to gain an understanding of visitors’ satisfaction with services, facilities, opportunities and overall satisfaction for evaluating program performance. Text mining was introduced in 2008 to analyze the unstructured data from customer comments on the survey. “THERE HAS GOT TO BE A BETTER WAY” When faced with manually summarizing and making sense of up to 2,000 observations of open-ended customer comments (i.e., unstructured data or qualitative data), a common response is “There has got to be a better way”. Typically the approach to analyze unstructured data is to manually read each record and assign category codes (i.e., coding). This is a labour intensive and frustrating task. It begs the question if there is software to treat text as data. SAS Text Miner can automate text categorization. It not only replaces the task of manual text categorization but also supports insight discovery through predictive and descriptive models. Equally important, the SAS solution improves data driven decision-making.